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We construct the generalized entropy optimized by a given arbitrary statistical 
distribution with a finite linear expectation value of a basic random quantity of interest. 
This offers, via the maximum entropy principle, a unified basis for a great variety of 
distributions observed in nature, which can hardly be described by the conventional 
methods. As a simple example, we explicitly derive the entropy associated with the 
stretched exponential distribution. To include the distributions with the divergent 
moments (e.g., the Levy stable distributions), it is necessary to modify the definition of 
the expectation value. 
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Diversity of statistical distributions observed in nature is truly remarkable. In 
particular, a number of distributions, which are anomalous in view of ordinary statistical 
mechanics, can be recognized in a variety of complex systems. To understand the 
properties of such complex systems deeper, it is desirable to characterize these 
distributions within a unified framework of the statistical principles. The maximum 
entropy principle can be thought of as such one [1]. 

In a physical experiment, what to be measured is quite often the distribution and not 
directly the entropy itself. Therefore, it is necessary to be able to find the corresponding 
entropic measure optimized by the observed distribution under appropriate constraints. 

In this paper, we present the entropy-generating algorithm and construct the 
generalized entropy, which can be optimized by a given arbitrary distribution with the 
finite linear expectation value of a random quantity of interest. As an example, we 
explicitly derive the entropy associated with the stretched exponential distribution. 

Let us start our discussion by considering a continuous function / of [eDcR, 
whose range is [0, 1] and is assumed to be integrable over D 



f need not be a monotonic function, in general. 

In terms of a probability distribution {/?,} ;=u ,...,w w i tn W the number of accessible 
states, we define the following quantity: 




(1) 



D 
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w 

A[p;t)=Y,(p-f(t)) 

1=1 



(2) 



In this equation, the symbol (x) + stands for 



(jc) + = max {0, x], 



(3) 



which satisfies 



(Ajf + a-A)y) + <AU) + + (l-A)(y) 



+ 



(VAe[0,l]), 



(4) 



(5) 



where 0(x) is the Heaviside unit step function defined by 0{x) = for x < and 
= 1 for ^ > 0. Though we are considering a discrete distribution, generalizations 
of the subsequent results to the continuous case are straightforward. In the case when 
the distribution is continuous, the range of / should be extended from [0, 1] to [0, °°) , 
in general. 

The quantity in Eq. (2) has some interesting properties. Among others, what we 
notice here are the following two: 



0<A[p;0<l, 



(6) 
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A[Xp + (l-X) p'; t) < XA[p\ t) + (X-X)A[p\ t) 



(VAe[0,l]). 



(7) 



The latter directly follows from Eq. (4). 
Next, we consider the integral 



/[/>] = J dt{l-A[p;t)), (8) 



which is manifestly a positive functional due to Eq. (6). Using the expression in Eq. (5), 
we find this integral to be rewritten as follows: 

w 

I[p]=Y,\ dt (Pi-f - e( Pi - f (t))\ + WF, (9) 

i=l o 

where F is given in Eq. (1). 

Our entropy-generating algorithm consists of essentially identifying the generalized 
entropy, 5, with the positive functional /: 

S[p] = kl[p] + a, (10) 

where k and a are constants, and in particular k has to be positive. Since the entropy 
should vanish for the completely ordered state, Pi ,= pf ] = 8 in (with n a natural 
number between 1 and W), the constant a satisfies the condition: kl[p (0) ] + a = 0. 
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Consequently, the generalized entropy is given by 



S[p] = k(l[p]-I[p (0) ]). 



(ID 



Note that this is, in fact, a concave functional: 



S[Xp + (l-X)p']>XS[p] + (l-X)S[p'], 



(12) 



as can be ascertained by using Eq. (7). 

Let us first discuss a relatively simple case when / is a monotonically decreasing 
function defined in D - [0, °°) . In this case, / in Eq. (9) can further be written as 



/i>]=£ 



i=l 



f'Hp,) 



Pj~\pd- \ dtf(t) 



+ WF, 



(13) 



where / 1 is the inverse function of / . Therefore, the generalized entropy reads 



S[p] = k 



i=i 



r x {p>) 

Pi f-\p)- \ dtf(t) 



+ c 



(14) 



where c is a constant given by 
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/ _1 (0) f-\i) 

c = W J" dtf(t)+ J" dtf(t)-f-\\). 

f-'(Q) 



(15) 



Let us employ S in eq. (14) as the entropy for the maximum entropy principle. 
Consider the functional 



where Q l is the zth value of the basic random variable, Q, and a and ft are the 
Lagrange multipliers associated with the constraints on the normalization condition and 
on the linear expectation value of Q denoted by < Q >, respectively. (Here, clearly the 
ordinary expectation value is assumed to be well defined. However, there exist 
distributions with no finite moments. Celebrated examples are the Levy stable 
distributions. To treat such distributions, it is necessary to modify the definition of the 
expectation value. See the later remarks.) Under the assumption of differentiability of 
f~ l , variation of <I> with respect to {/?,} i=1 i2 ,-,w l ea ds to the following stationary 
distribution: 



provided that the positive constant, k, has been eliminated by rescaling the Lagrange 
multipliers. a can be determined by the normalization condition: 




&[p:a,p\ = S[p]-a l>,£,-<0> , 



(16) 



V /=i ) \i=\ J 




(17) 
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£ f(a + PQi) = l. Therefore, an arbitrary monotonically decreasing distribution 

with the finite linear expectation value of Q could be derived from the maximum 
entropy principle. 

The above discussion can immediately be applied to the stretched exponential 
distribution. In this case, / is taken to be 

/(0 = exp(-^) ae[0,oo), ye (0,1)). (18) 

Then, the associated generalized entropy is found to be given by 

w 1 

S[/>]=£ r(l + l/y,-ln/> ( .)--r(l/y), (19) 

i=l 7 

where T(s,x) is the incomplete gamma function of the second kind defined by 
T(s,x)= | dtt"~ l e~* , and r(,y) = r(s, 0) is the gamma function. The generalized 

entropy in Eq. (19) is identical to the one recently discussed in Ref. [2], in which it 
seems to have come from the sky. 

We mention that, taking the limit y — > 1 - in the above discussion, / becomes the 
exponential function of t and Eq. (19) converges to the ordinary Boltzmann-Gibbs- 
Shannon entropy, as it should do [3]. It can be shown [4] that if / is taken to be the q- 
exponential function [5,6], then the corresponding generalized entropy is the Tsallis 
entropy [7]. Similarly, the so-called K -deformed entropy [8,9] can also be derived if 
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the K - deformed exponential function is employed (though we do not present its 
explicit derivation here). 

Now, let us succinctly discuss the case when / in Eq. (2) is not monotonic. In such a 
case, the domain interval D of / should be divided into the subintervals, {D a }, in 
which / is monotonic. D is now the disjoint union of D a 's. Accordingly, Eqs. (8) 
and (9) are written as 

/[/>] = £ J dt{\-A[p;t)), (20) 

a D a 

and 

w 

™ = LL j dt( Pi -f(t))[l-e( Pi -f(t))] + WF, (21) 

a i'=l Q a 

respectively. Let D* = D*|/ _1 (p,Jj be the subinterval of D a , in which Pi<f{t). 
Then, Eq. (21) can further be rewritten as follows: 



a i=\ 



Li\pii(D;(r'( Pl )))-F;(r( Pi )) 



a i'=l 



(22) 
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where l\D* a j is the length of the interval D a and F* = j dt f(t). Substitution of 

this quantity into Eq. (11) leads to the generalized entropy optimized by a non- 
monotonic distribution. 

Finally, we mention that the Legendre transform structure highlighted by the relation 

JT7T = P (23) 
d<Q> 

exists in the present generalized theory. This is due to the fact [10,11] that Eq. (23) 
holds for an arbitrary form of the entropy and an arbitrary definition of the expectation 
value. 

In conclusion, we have constructed the generalized entropy, which is optimized by 
any given statistical distribution. This can be regarded as an extreme generalization of 
the maximum entropy principle. The present approach assumes that the distribution has 
a finite linear expectation value of a basic random quantity of interest. If the lowest 
moments are divergent, then the definition of the expectation value should be modified. 
For example, in Tsallis statistics [3-5], the optimal distribution is the so-called q- 
exponential distribution, which can asymptotically be a power-law distribution. In this 
case, the expectation value should be defined in terms of the escort distribution [12]. 
Regarding necessity of modifying the definition of the expectation value, the 
approaches from the different perspectives are needed [13]. 
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